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© Method and apparatus for coding moving image signal. 



© According to a method and apparatus for coding 
a moving image signal, a change between a current 
frame and a previous frame is detected in units of 
blocks obtained by dividing a frame constituted by 
an input moving image signal into a plurality of 
pixels. A first valid block map is formed in units of 
frames by determining valid and invalid blocks on 
the basis of the detected changes. A first weighting 
operation of the formed first valid block map is 
performed. A second weighting operation of a fourth 
valid block map of the previous frame is performed. 



A weighted second valid block map is obtained by 
adding/synthesizing the first and fourth valid block 
maps respectively subjected to the first and second 
weighting operations. A third valid block map is 
obtained by performing predetermined segmentation 
of the second valid block map. A fourth valid block 
map is obtained by determining validity of an iso- 
lated invalid block in the third valid block map by 
referring to neighboring blocks. A valid block region 
of the fourth valid block map is coded. 
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The present invention relates to a method and 
apparatus for coding a moving image signal by 
using a band compression technique. 

As a conventional scheme for coding a moving 
image signal by using a band compression tech- 
nique, for example, a scheme described in "Color 
Moving Image Videophone Compatible with ISDN", 
Spring National Convention of the Institute of Elec- 
tronics and Communication Engineers of Japan, 
Reference No. D-233, 1989 is known. In this 
scheme for coding a moving image signal, a face 
region is extracted from a frame to form a map. In 
an image coding section, inter-frame and intra- 
frame adaptive prediction operations are per- 
formed. In this case, if a face region is determined, 
coding is performed to the last step. Otherwise, 
coding is stopped in a step immediately preceding 
the last step to reduce the coding amount 

In the above-described conventionaJ scheme 
for coding a moving image signal, however, since a 
background portion other than a face portion is 
coarsely coded, unnecessary information is gen- 
erated because of noise in the background portion. 
In addition, if a background portion is changed to a 
face portion between continuous frames, since a 
coarsely coded portion is changed to a finely cod- 
ed portion, a large number of predictive error sig- 
nals are inevitably generated. That is, unnecessary 
information is coded. As a result, the coding effi- 
ciency is decreased. 

Summary of the Invention 

It is an object of the present invention to pro- 
vide a method and apparatus for coding a moving 
image signal, which can improve the coding effi- 
ciency. 

It is another object of the present invention to 
provide a method and apparatus for coding a mov- 
ing image signal, which can remove unnecessary 
information generated by noise in a background 
portion. 

It is still another object of the present invention 
to provide a method and apparatus for coding a 
moving image signal, which codes only a speaker 
region. 

In order to achieve the above objects, accord- 
ing to an aspect of the present invention, there is 
provided a method of coding a moving image sig- 
nal, comprising the steps of detecting a change 
between a current frame and a previous frame in 
units of blocks obtained by dividing a frame con- 
stituted by an input moving image signal into a 
plurality of pixels, forming a first valid block map in 
units of frames by determining valid and invalid 
blocks on the basis of the detected changes, per- 
forming a first weighting operation of the formed 
first valid block map, performing a second weigh- 



ting operation of a fourth valid block map of the 
previous frame, obtaining a weighted second valid 
block map by adding/synthesizing the first and 
fourth valid block maps respectively subjected to 

s the first and second weighting operations, obtaining 
a third valid block map by performing predeter- 
mined segmentation of the second valid block map, 
obtaining a fourth valid block map by determining 
validity of an isolated invalid block in the third valid 

10 block map by referring to neighboring blocks, and 
coding a valid block region of the fourth valid block 
map. 

According to another aspect of the present 
invention, there is provided an apparatus for coding 

ts a moving image signal, comprising forming means 
for detecting a change between a current frame 
and a previous frame in units of blocks obtained by 
dividing a frame constituted by an input moving 
image signal into a plurality of pixels, and forming 

20 a first valid block map in units of frames by deter- 
mining valid and invalid blocks on the basis of the 
detected changes, first weighting means for per- 
forming a first weighting operation of the formed 
first valid block map, second weighting means for 

25 performing a second weighting operation of a 
fourth valid block map of the previous frame, add- 
ing means for obtaining a weighted second valid 
block map by adding/synthesizing outputs from the 
first and second weighting means, segmentation 

30 means for obtaining a third valid block map by 
performing predetermined segmentation of the sec- 
ond valid block map, isolated invalid block remov- 
ing means for obtaining a fourth valid block map by 
determining validity of an isolated invalid block 

35 output from the segmentation means by referring 
to neighboring blocks, and coding means for cod- 
ing a valid block region output from the isolated 
invalid block removing means. 

In a videophone and the like, while a back- 

40 ground portion remains stationary, a speaker main- 
ly moves. Therefore, if a speaker portion is ex- 
tracted and subjected to coding, unnecessary in- 
formation generated by noise in the background 
and the like can be removed, and the coding 

45 efficiency can be improved. 

In the present invention, the differences be- 
tween frames are detected, and segmentation 
(linkage and removal of motion regions) is per- 
formed with respect to each portion corresponding 

so to the difference equal to or larger than a predeter- 
mined value, thus improving the coding efficiency. 
A method of extracting a speaker region by a 
difference detection scheme according to an em- 
bodiment of the present invention will be described 

55 in detail below with reference to the accompanying 
drawings. Assume that a speaker moves as in- 
dicated by the frames at time tO. time tl, and time 
t2 in Fig. 1. In addition, assume that when the 
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differences between the frames at time t1 and time 
t2, the shaded portions shown in Fig. 2 are ob- 
tained, and that the isolated shaded portions in the 
background portion correspond to difference sig- 
nals generated by noise in the background. Subse- 
quently, each frame is divided into a plurality of 
blocks, each consisting of n horizontal pixels x n 
vertical pixels. If the sum of~"the absolute values of 
difference signals in each block is equal to or 
larger than the first threshold value, the corre- 
sponding block is defined as a valid block. If the 
sum of the absolute values of the difference signals 
is less than the first threshold value, the corre- 
sponding block is defined as an invalid block. The 
first valid block map obtained by the above-de- 
scribed processing is shown in Fig. 3B. The 
shaded portions in Fig. 3B are valid blocks. As- 
sume that Fig. 3A shows the fourth valid block map 
obtained between the frames at time tO and time 
t1 . The first and second weighting operations of the 
valid block map of the current frame, i.e., the first 
valid block map and the valid block map of the 
previous frame, i.e., the fourth valid block map are 
respectively performed. An example of weighting 
will be described below. For example, valid and 
invalid blocks of the previous frame are respec- 
tively represented by n 1" and "0". A valid block of 
the current frame is represented by "2 n , and an 
invalid block thereof is represented by "0", similar 
to an invalid block of the previous frame. The 
second valid block map is obtained by 
adding/synthesizing the weighted valid block maps 
of the previous and current frames. Fig. 4A shows 
the second valid block map. 

Subsequently, segmentation of the second valid 
block map obtained by the above 
adding/synthesizing operation is performed. An ex- 
ample of segmentation will be described below 
with reference to 4A to 5. Assume that a block k in 
Fig. 5 is a target segmentation block. In this case, 
the values of blocks a, b, c, d t e, f, g, and h 
adjacent to the block k~are referred. More specifi- 
cally, the value of the second valid block map 
shown in Fig. 4A is referred. If the values of the 
neighboring blocks a, b. c, d, e, f, g, and h are 
equal to or larger than a predetermined second 
threshold value, the target block k is regarded as a 
valid block. If the values of the neighboring blocks 
a, b f c, d, e, f, g, and h are less than a predeter- 
mined" second" th7eshold~vaIue, the target block k is 
regarded as an invalid block. 

The third valid block map newly obtained by 
segmentation is shown in Fig. 4B. An isolated 
invalid block x is sometimes formed in the third 
valid block map. In this case, if coding of only the 
valid block region in the third valid block map is 
executed, the isolated invalid block x in the valid 
block region is not coded. As a result, coding 



distortion occurs in the invalid block x. and the 
resulting coded image greatly deteriorates. For this 
reason, removal of the isolated invalid block x is 
performed. In a method of removing the isolated 

5 invalid block x, processing similar to segmentation 
of an isolated invalid block is performed. More 
specifically, the values of blocks adjacent to the 
isolated invalid block x are referred. If the values of 
the neighboring blocks are equal to or larger than a 

70 predetermined threshold value, the isolated invalid 
block x as a target is replaced with a value repre- 
senting a valid block. With the above-described 
processing, the region of the isolated invalid block 
x in Fig. 4B is removed to obtain the fourth valid 

rs block map. Fig. 6 shows the fourth valid block map. 
Subsequently, a portion, of the input moving image 
signal, which corresponds to the valid block region 
in Fig. 6, i.e., the speaker region, is coded by using 
either or both of inter-frame correlation and intra- 

20 frame correlation. With this operation, unnecessary 
information generated by noise in the background 
and the like can be easily removed, and the coding 
efficiency can be improved. 

As the above-mentioned threshold values and 

25 the values for weighting, statistically obtained op- 
timal values are used. In addition, the arrangement 
and number of reference pixels in segmentation 
and isolated invalid block removing operation are 
not limited to those described above. 

30 

Brief Description of the Drawings 

Figs. 1 to 6 are views showing a method of 
coding a moving image signal according to an 

35 embodiment of the present invention; 

Fig. 7 is a block diagram showing an apparatus 
for coding a moving image signal according to 
the embodiment shown in Figs. 1 to 6; 
Figs. 8 to 10 are views showing a method of 

40 coding a moving image signal according to an- 
other embodiment of the present invention; 
Fig. 1 1 is a block diagram showing an apparatus 
for coding a moving image signal according to 
the embodiment shown in Figs. 8 to 10; 

45 Figs. 12 to 14 are block diagrams respectively 
showing a segmentation section, an isolate in- 
valid block removing section, and a coding sec- 
tion; 

Figs. 15 to 19D are views for explaining a meth- 
so od of coding a moving image signal according 
to still another embodiment of the present inven- 
tion; 

Fig. 20 is a block diagram showing an apparatus 
for coding a moving image signal according to 
55 the embodiment shown in Figs. 15 to 19D; 

Figs. 21 and 22 are block diagrams respectively 
showing a valid block count determining section 
and a valid block reset section; 
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Fig. 23 is a block diagram showing an apparatus 
for coding a moving image signal according to 
the embodiment shown in Figs. 15 to 19D; 
Figs. 24 to 31 D are views for explaining a meth- 
od of coding a moving image signal according 
to still another embodiment of the present inven- 
tion; 

Figs. 32 to 34 are block diagrams respectively 
showing apparatuses for coding a moving image 
signal according to the embodiment shown in 
Figs. 24 to 31 D; 

Figs. 35, 37, 38, and 40 are block diagrams 
respectively showing modifications of the ap- 
paratus for coding a moving image signal ac- 
cording to the present invention; 
Fig. 36 is a block diagram showing another 
arrangement of the coding section; and 
Fig. 39 is a block diagram showing a ratio 
determining section. 

Description of the Preferred Embodiments 

An apparatus based on a difference detection 
scheme according to an embodiment of the 
present invention will be described below with ref- 
erence to the accompanying drawings. Fig. 7 
shows an embodiment of the present invention. An 
input moving image signal is supplied to a validity 
determining section 1 and a delay section 8 
through a line 10. The validity detecting section 1 
stores a moving image signal of a previous frame, 
and obtains a frame difference signal based on the 
difference between the moving image signal of the 
previous frame and the new moving image signal 
input through the line 10. In addition, the validity 
detecting section 10 divides this frame difference 
signal into blocks, each consisting of a plurality of 
pixels, i.e., n horizontal pixels x n vertical pixels, 
and obtains" the sum of the absolute values of 
frame difference values in each block. If the ob- 
tained sum of the absolute frame difference values 
in a give block is equal to or larger than a first 
threshold value, it is determined that the block is 
valid. If the sum of the absolute frame difference 
values in a given block is less than the first thresh- 
old value, it is determined that the block is invalid. 
In this manner, the validity determining section 1 
obtains a first valid block map. The first valid block 
map obtained by the validity determining section 1 
is supplied to a weighting section 2. The weighting 
section 2 performs a predetermined first weighting 
operation of the first valid block map supplied from 
the validity determining section 1. The first valid 
block map weighted by the weighting section 2 is 
supplied to an adder 4. The adder 4 adds the first 
valid block map supplied from the weighting sec- 
tion 2 to a fourth valid block map, of the previous 
frame, supplied from a weighting section 3, thus 



obtaining a second valid block map. The second 
valid block map obtained by the adder 4 is sup- 
plied to a segmentation section 5. The segmenta- 
tion section 5 performs segmentation processing of 

5 all the blocks in the second valid block map sup- 
plied from the adder 4. Assume that a block k is a 
target block for segmentation, as shown in Rg. 5. 
In this case, the values of the block k and neigh- 
boring blocks a, b, c, d, e, f, g, and h are referred. 

to If these values~are equal To oflarger Than a second 
predetermined threshold value, it is determined that 
the block k is valid. If the values are less than the 
second thFeshold value, it is determined that the 
block k is invalid. In this manner, the segmentation 

15 section 5 obtains a third valid block map. The third 
valid block map obtained by the segmentation sec- 
tion 5 is supplied to an isolated invalid block re- 
moving section 6. The isolated invalid block remov- 
ing section 6 performs an isolated invalid block 

20 removing operation of invalid blocks included in the 
third valid block map, and links the valid blocks 
with each other. In processing for isolated invalid 
blocks, similar to segmentation processing, blocks 
near an isolated invalid block as a target are re- 

25 ferred. If the values of the neighboring blocks are 
equal to or larger than a third predetermined 
threshold value, it is determined that the isolated 
invalid block is a valid block. If the values of the 
neighboring blocks are less than the third threshold 

30 value, it is determined that the isolated invalid 
block is an invalid block. With the above-described 
operation, a fourth valid block map from which 
isolated invalid blocks are removed is obtained. 
The fourth valid block map obtained by the isolated 

35 invalid block removing section 6 is supplied to the 
weighting section 3 and a coding section 7. The 
weighting section 3 performs a second weighting 
operation of the fourth valid block map supplied 
from the isolated invalid block removing section 6, 

40 and supplies the weighted fourth valid block map to 
the adder 4. A delay section 8 performs delay time 
compensation of the input moving image signal by 
an amount corresponding to the time interval be- 
tween the instant at which the input moving image 

45 signal is supplied to the delay section 8 and the 
instant at which the fourth valid block map is sup- 
plied to the coding section 7, thus synchronizing 
the fourth valid block map with the input moving 
image signal. The time-compensated moving im- 

50 age signal output from the delay section 8 is sup- 
plied to the coding section 7. The coding section 7 
codes a portion, of the moving image signal sup- 
plied from the delay section 8, which corresponds 
to a valid block region, i.e.. a speaker region, in the 

55 fourth valid block map supplied from the isolated 
invalid block removing section 6, but does not code 
a background portion indicated by invalid blocks. 
As a coding method, one of the following meth- 
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ods is used: a method of using inter-frame correla- 
tion, e.g., motion compensation, a method of using 
intra-frame correlation, e.g., orthogonal conversion, 
and a method of using both inter-frame correlation 
and intra-frame correlation. 

As the above-mentioned threshold values, sta- 
tistically obtained optimal values are used. 

A method of extracting a speaker portion using 
the difference detection scheme according to an- 
other embodiment will be described next with refer- 
ence to the accompanying drawings. This method 
employs a segmentation method different from that 
in the above-described method. Assume that after 
a speaker moves, as indicated by the frames at 
time tO, time t1, and time t2 in Fig. 1, and the 
speaker stops his/her motion but moves only 
his/her mouth. In this case, if the differences be- 
tween the frames at time t1 and time t2 are ob- 
tained, hatched regions in Fig. 2 are obtained. The 
differences between the frames at time t2 and time 
t3 correspond to hatched regions in Fig. 8A. As- 
sume that the isolated hatched portion in the back- 
ground portion is a difference signal generated by 
noise in the background. 

Fig. 8B shows a valid block map between time 
t2 and time t3, which is obtained by comparing a 
first threshold value with the sum of the absolute 
values of each difference signal. Referring to Fig. 
8B, shaded portions indicate valid blocks. 

A case will be described below, in which seg- 
mentation based on the method of this embodi- 
ment is performed with respect to a second valid 
block map shown in Fig. 4A which is obtained by 
an adding/synthesizing operation. If the ratio of a 
valid block count as a denominator in the fourth 
valid block map as a previous frame to a valid 
block count as a numerator in the first valid block 
as a current frame is equal to or larger than a 
fourth predetermined threshold value, e.g., if the 
valid block count of the current frame is substan- 
tially equal to or 1/2 or more the valid block count 
of the previous frame, the fourth valid block is 
obtained by the segmentation method described 
with reference to Figs. 4A to 5. 

Assume that the ratio of the valid block count 
of the previous frame to the valid block count of 
the current frame is less than the fourth threshold 
value, and the valid block count of the current 
frame is much smaller than the valid block count of 
the previous frame, similar to the case at time t3, 
e.g., less than 1/2. Segmentation processing in this 
case will be described below with reference to 
Figs. 6, 8, 9, and 10. When the first and second 
weighting operations of the fourth valid block map 
obtained at time t2 in Fig. 6 and the first valid block 
map at time t2 in Fig. 8B are respectively per- 
formed, and the resulting maps are synthesized 
with each other, the second block map shown in 



Fig. 9A is obtained. When segmentation of the 
second valid block map is performed on the basis 
of the second threshold value, a third valid block 
map indicated by a hatched portion in Fig. 9B is 

5 obtained. Isolated invalid blocks are removed from 
the third valid block map on the basis of the third 
threshold value to obtain a fourth valid block map. 
In this case, since no isolated invalid blocks are 
present in the third valid block map obtained at 

70 time t3. the fourth valid block map is equivalent to 
the third valid block map. 

Subsequently, coding is performed with respect to 
only a portion, of the input moving image signal, 
which corresponds to the valid block region in the 

75 fourth valid block map. However, since the chest 
portion and upper right portion of the head of the 
speaker are omitted from the valid block region of 
the fourth valid block map, if coding is performed 
in this state, non-coded regions are generated in 

20 the chest portion and the head portion. As a result, 
discontinuous portions are formed in the speaker 
region of a coded image, and the coded image 
may deteriorate. If, therefore, the ratio of the valid 
block count of the previous frame to the valid block 

25 count of the current frame is less than the fourth 
threshold value, and the valid block count of the 
current frame is small, similar to the case at time 
t3, omissions from the speaker region are pre- 
vented by switching the threshold value in seg- 

30 mentation to another value. Assume that segmenta- 
tion is executed in the weighted second valid block 
map shown in Fig. 9A. In this case, if, for example, 
the block k in Fig. 5 has a value other than "0", the 
threshold value is set to be small enough to make 

35 the block k as a segmentation target block valid, 
thus obtaining the third valid block shown in Rg. 10 
and preventing omissions from the speaker region. 
The threshold value in this case is defined as a fifth 
threshold value. As described above, if the ratio of 

40 the valid block count of the previous frame to the 
valid block count of the current frame is equal to or 
larger than the fourth threshold value, segmentation 
based on the second threshold value is performed. 
If this ratio is smaller than the fourth threshold 

45 value, and the valid block count of the current 
frame is much smaller than that in the previous 
frame, segmentation based on the fifth threshold 
value is performed. Subsequently, coding of the 
input moving image signal corresponding to the 

so valid block region, i.e., the speaker region, shown 
in Rg. 6 or 10, is performed by using either or both 
of inter-frame correlation and intra-frame correla- 
tion. 

Rg. 11 shows an apparatus based on the dif- 
55 ference detection scheme according to the em- 
bodiment shown in Rgs. 8 to 10. The same refer- 
ence numerals in Rg. 1 1 denote the same parts as 
in Rg. 7, and a detailed description thereof will be 
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omitted. 

A ratio determining section 8 determines the 
ratio of a valid block count of the fourth vaJid block 
map of a previous frame supplied from an isolated 
invalid block removing section 6 to a valid block 
count of the first valid block map supplied from a 
validity determining section 1, and supplies to a 
segmentation section 5 a determination signal in- 
dicating whether the determination result is not less 
than a predetermined fourth threshold value or less 
than it The segmentation section 5 performs seg- 
mentation processing of the all the block in the 
second valid block map supplied from an adder 4. 
If, for example, the determination signal supplied 
from the ratio determining section 8 indicates that 
the rate is equal to or larger than the fourth thresh- 
old value, the segmentation section 5 selects a 
second threshold value as a threshold value in 
segmentation. If the determination signal indicates 
that the rate is less than the fourth threshold value, 
the segmentation section 5 selects a fifth threshold 
value as a threshold value in segmentation, thus 
performing segmentation. Assume that a block k is 
a target block for segmentation, as shown in Fig. 5. 
In this case, the values of the block k and neigh- 
boring blocks a, b, c, d, e, f, g, and h are referred. 
If these values~are equal to or larger than a thresh- 
old value selected by a determination signal sup- 
plied from the ratio determining section 8, it is 
determined that the block k is valid. If the values 
are less than the selected threshold value, it is 
determined that the block k is invalid. In this man- 
ner, the segmentation section 5 obtains a third 
valid block map. The third valid block map ob- 
tained by the segmentation section 5 is supplied to 
the isolated invalid block removing section 6. Sub- 
sequently, processing similar to those in the above- 
described embodiment is performed until coding is 
performed. 

Fig. 12 shows an arrangement of the seg- 
mentation section 5. The segmentation section 5 
can be constituted by line memories 50 and 51, 
delay circuits 52, 53, 54, 55, 56, and 57, and a 
ROM 58. The second valid block map output from 
the adder 4 is supplied to the line memory 50, the 
delay circuit 52, and the ROM 58 through a line 45. 
The line memory 50 delays the valid block map 
supplied through the line 45 by an amount cor- 
responding to one block line, and supplies the 
delayed map to the line memory 51, the delay 
circuit 54, and the ROM 58. The line memory 51 
delays the signal supplied from the line memory 50 
by an amount corresponding to one block line, and 
supplies the delayed signal to the delay circuit 56 
and the ROM 58. Each of the delay circuits 52. 53, 
54. 55, 56, and 57 delays the supplied signal by 
one clock and supplies it to the ROM 58 and the 
next delay circuit. An output signal from the delay 



circuit 54 corresponds to the block k as a seg- 
mentation target block. The ROM 58 writes a value 
representing a valid block in an address area in 
which a signal supplied from one of the line memo- 

5 ries or one of the delay circuits is equal to or larger 
than the second threshold value, and writes a value 
representing an invalid block in an address area in 
which the value of the supplied signal is smaller 
than the second threshold value. By using the line 

to memories, the delay circuits, and the ROM in the 
above-described manner, the values of the seg- 
mentation target block k and the neighboring 
blocks can be referred, "and the result can be 
output through a tine 580. Note that a line 85 is a 

is line through which a signal is supplied from a ratio 
determining section 9 in an apparatus shown in 
Fig. 33. The isolated invalid block removing section 
6 will be described next with reference to Fig. 13. 
Similar to the segmentation section 5, the isolated 

20 invalid block removing section 6 comprises line 
memories 60 and 61, delay circuits 62, 63, 64. 65, 
66, and 67, and a ROM 68. The isolated invalid 
block removing section 6 removes an isolated in- 
valid block by referring to the values of blocks 

25 adjacent to the invalid block. A signal supplied from 
the delay circuit 64 to the ROM 68 corresponds to 
a target block k subjected to an isolated invalid 
block removing~operation. If the signal supplied 
from the delay circuit 64 to the ROM 68 represents 

30 an invalid block, and the value of each neighboring 
block is equal to or larger than a third predeter- 
mined threshold value, the ROM 68 outputs a value 
representing a valid block. In contrast to this, if the 
signal supplied from the delay circuit 64 represents 

35 an invalid block, and the value of each neighboring 
block is less than the third threshold value, the 
ROM 68 outputs a value representing an invalid 
block. In addition, if the signal supplied from the 
delay circuit 64 represents a valid block, the ROM 

ao 68 outputs a value representing a valid block. The 
above-described logic is written in the ROM 68 in 
advance to remove isolated invalid blocks and link 
valid blocks to each other. The fourth valid block 
map output from the isolated invalid block remov- 
es ing section 6 is supplied to a weighting section 3 
and a coding section 7 through a line 680. The 
weighting section 3 can be constituted by a ROM, 
similar to the weighting section 2. In this case, a 
value of second weighting is written in the ROM in 

so advance. Fig. 14 shows the arrangement of the 
coding section 7. A moving image signal supplied 
from a delay circuit 8 to the coding section 7 
through a line 870 is supplied to a motion vector 
detector 71 and a subtracter 72 in the coding 

55 section 7. The fourth valid block map output from 
the isolated invalid block removing section 6 is 
supplied, as a signal representing a region for 
which coding is executed, to a quantizer 74 
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through a line 687. The motion vector detector 71 
stores the signal of a previous frame, and detects a 
motion between the signal of the previous frame 
and a new signal input through the line 870, thus 
supplying a motion vector representing the amount 
and direction of each motion to a frame memory 77 
and a variable-length coder 78. The subtracter 72 
performs a subtraction of the input signal supplied 
through the line 870 and a motion-compensated 
predictive signal supplied from the frame memory 

77 to obtain a motion compensation predictive er- 
ror signal. The motion compensation predictive er- 
ror signal obtained by the subtracter 72 is supplied 
to an orthogonal converter 73. The orthogonal con- 
verter 73 performs orthogonal conversion of the 
motion compensation predictive error signal sup- 
plied from the subtracter 72 to convert the predic- 
tive error signal in a spatial region into a predictive 
error signal in a frequency region. The predictive 
error signal in the frequency region output from the 
orthogonal converter 73 is supplied to the quantizer 
74. The quantizer 74 quantizes each predictive 
error signal, supplied from the orthogonal converter 
73, which corresponds to a block indicated as a 
valid block by the fourth valid block map supplied 
through the line 687. With regard to a block in- 
dicated as an invalid block by the fourth valid block 
map supplied through the line 687, the quantizer 
74 outputs "0" to stop coding. The output signal 
from the quantizer 74 is supplied to an inverse 
orthogonal converter 75 and the variable-length 
coder 78. The inverse orthogonal converter 75 per- 
forms inverse orthogonal conversion of the predic- 
tive error signal supplied from the quantizer 74 to 
convert it into the predictive error signal in the 
spatial region again. The output signal from the 
inverse orthogonal converter 75 is supplied to an 
adder 76. The adder 76 adds the predictive error 
signal in the spatial region, supplied from the in- 
verse orthogonal converter 75, to the motion-com- 
pensated predictive signal supplied from the frame 
memory 77, thus obtaining a local decoded signal. 
The local decoded signal output from the adder 76 
is supplied to the frame memory 77. The frame 
memory 77 changes the delay amount of the local 
decoded signal supplied from the adder 76 in ac- 
cordance with the motion vectors supplied from the 
moving vector detector 71 to obtain a motion- 
compensated predictive signal. The motion-com- 
pensated predictive signal output from the frame 
memory 77 is supplied to the subtracter 72 and the 
adder 76. Subsequently, the variable-length coder 

78 converts the quantized predictive error signal in 
the frequency region supplied from the quantizer 
74 and the motion vectors supplied from the mo- 
tion vector detector 71 into variable-length codes 
by using an efficient coding scheme such as Huff- 
man coding, thus reducing the redundancy. The 



variable-length codes whose redundancy is re- 
duced are output from the variable-length coder 78 
to be output from the coding section 7 to a trans- 
mission line, while the coding rate and the speed of 
5 the transmission line are synchronized with each 
other. 

With this operation, therefore, coding is per- 
formed with respect to only a valid block region, 
i.e., a speaker region, in accordance with the fourth 

10 valid block map, but coding is stopped with respect 
to a background portion indicated by an invalid 
block. Coding may be stopped by setting the out- 
put of the subtracter 72 at "0" level. 

A method of extracting a speaker region ac- 

T5 cording to still another embodiment will be de- 
scribed in detail below with reference to the ac- 
companying drawings. Assume that a speaker 
moves as indicated by the frames at time tO, time 
t1, and time t2 in Fig. 15. The hatched regions 

20 shown in Fig. 16 correspond to the differences 
between the frames at time tO and time t1. The 
isolated hatched portion in the background portion 
in Fig. 16 corresponds to a difference signal gen- 
erated by noise in the background. Fig. 17B shows 

25 a valid block map at time t1 which is obtained by 
the same processing as described above. Each 
shaded portion in Fig. 17B is a valid block. Fig. 
17A shows the sixth valid block map obtained 
between the frame at time tO and the frame at time 

30 tO-1 immediately preceding the frame at time tO. 
The first weighting operation of the valid block map 
of the current frame (Fig. 17B), i.e., the first valid 
block is performed, whereas the second weighting 
operation of the valid block map of the previous 

35 frame (Fig. 17A), i.e., the sixth valid block map is 
performed. The weighted valid block maps of the 
current and previous frames are added/synthesized 
to obtain the second valid block map. Fig. 17C 
shows the second valid block map. Subsequently, 

40 segmentation is performed with respect to the sec- 
ond valid block map, as shown in Figs. 17C and 5. 

Fig. 17D shows the third valid block map ob- 
tained by the segmentation. If an isolated invalid 
block x shown in Fig. 18 is generated in the third 

45 valid block map, the region of the isolated invalid 
block is removed by the same processing as de- 
scribed above to obtain the fourth valid block map. 
Fig. 17D shows the fourth valid block map. 

Processing at time t2 will be described below. 

so When differences between the frames at time t1 
and time t2 are obtained, and validity determination 
is performed in accordance with the first threshold 
value, the first valid block map shown in Fig. 19A is 
obtained. The first weighting operation of the first 

55 valid block map is performed. Since Rg. 17D 
shows the valid block map at time tl as the pre- 
vious frame, the second weighting operation of the 
valid block map in Fig. 17D is performed. When 
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the weighted valid block map in Fig. 17D is 
added/synthesized using the first valid block map 
in Fig. 19B, for which the first weighting operation 
is performed, the second valid block map shown in 
Fig. 19B is obtained. When the above-mentioned 
segmentation is performed with respect to the sec- 
ond valid block map in Fig. 19B, the third valid 
block map shown in Fig. 19C is obtained. Subse- 
quently, an isolated invalid block removing opera- 
tion of the third valid block map is performed. 
Since no isolated invalid block is present in the 
third valid block in Fig. 19C. the third valid block 
map is used as the fourth valid block without any 
modification, thus obtaining a speaker region by 
segmentation. Although the actual speaker region 
at time t2 corresponds to substantially the left half 
of the frame, a large portion of the speaker region 
obtained by segmentation extends to the back- 
ground portion in the right half of the frame. If, 
therefore, the fourth valid block map in Fig. 19C is 
used without any modification, noise in the back- 
ground may be undesirably coded. That is, when 
the motion of a speaker is large as in the case of 
the frames at time t1 and time t2, and the number 
of valid blocks obtained by segmentation is large, 
the speaker region of the current frame extends 
into the speaker region of the previous frame due 
to the influences of the valid block map of the 
previous frame. For this reason, if the motion of a 
speaker between frames is large, i.e., the number 
of valid blocks in the fourth valid block map is 
equal to or larger than a fourth predetermined 
threshold value, all the valid blocks in the fourth 
valid block map are replaced with invalid blocks to 
obtain the fifth valid block map. The fifth valid block 
map is delayed by a one-frame time to become the 
sixth valid block map. At the next timing, the sixth 
valid block map is subjected to segmentation. As- 
sume that Figs. 17A and 17B respectively show the 
fourth block map of the previous frame and the 
valid block map of the current frame, i.e., the first 
valid block map. In this case, if the number of valid 
blocks in the fourth valid block map obtained at 
time t1 is equal to or larger than the fourth thresh- 
old value, all the valid blocks in the fourth valid 
block map are replaced with invalid blocks to ob- 
tain the fifth valid block map. Therefore, all the 
blocks in the sixth valid block map at time t2, 
which is obtained by delaying the fifth valid block 
map by a one-frame time, are invalid blocks. As a 
result, the weighted second valid block map shown 
in Fig. 19D is obtained, provided that the valid 
block map shown in Fig. 19A is the first valid block 
map at time t2. When the above-mentioned seg- 
mentation is performed with respect to this second 
valid block map, the third valid block map shown in 
Fig. 19A is obtained. Since no isolated invalid block 
is included in the third valid block map, the third 



valid block map is used as the fourth valid block 
map without any modification, thus omitting the 
background portion. 

Fig. 20 shows an apparatus based on the dif- 

5 ference detection scheme according to the em- 
bodiment shown in Figs. 15 to 19D. The same 
reference numerals in Fig. 20 denote the same 
parts as in Fig. 7, and a detailed description thereof 
will be omitted. 

io The fourth valid block map obtained by an 

isolated invalid block removing section 6 is sup- 
plied to a valid block count determining section 10, 
a valid block reset section 1 1 , and a coding section 
7. If the number of valid blocks in the fourth valid 

75 block map supplied from the isolated invalid block 
removing section 6 is equal to or larger than a 
fourth predetermined threshold value, the valid 
block count determining section 10 supplies a reset 
execution command to the valid block reset section 

20 1 1 . In contrast to this, if the number of valid blocks 
in the fourth valid block map supplied from the 
isolated invalid block removing section 6 is less 
than the fourth threshold value, the valid block 
count determining section 10 supplies a reset stop 

25 command to the valid block reset section 11. In 
response to the reset execution command from the 
valid block count determining section 10, the valid 
block reset section 1 1 replaces all the valid blocks 
in the fourth valid block map supplied from the 

30 isolated invalid block removing section 6 with in- 
valid blocks, thus obtaining the fifth valid block 
map. In contrast to this, in response to the reset 
stop command from the valid block count deter- 
mining section 10, the valid block reset section 11 

35 performs no processing with respect to the fourth 
valid block map supplied from the isolated invalid 
block removing section 6 to use the fourth valid 
block map as the fifth valid block map without any 
modification. The fifth valid block map obtained by 

40 the valid block reset section 11 is supplied to a 
frame delay section 1 2. The frame delay section 1 2 
delays the fifth valid block map supplied from the 
valid block reset section 1 1 by a one-frame time to 
obtain the sixth valid block map. The sixth valid 

45 block map obtained by the frame delay section 12 
is supplied to a weighting section 3. The weighting 
section 3 performs the second weighting operation 
of the sixth valid block map supplied from the 
frame delay section 12, and supplies the weighted 

so fourth valid block map to an adder 4. 

Fig. 21 shows an arrangement of the valid 
block count determining section 8. The fourth valid 
block map obtained by the isolated invalid block 
removing section 6 is supplied to a counter 81 in 

55 the valid block count determining section 10 
through a line 680. The counter 81 counts valid 
blocks in the fourth valid block map supplied from 
the isolated invalid block removing section 6, and 
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supplies the valid block count to a comparator 82. 
The fourth threshold value used for threshold deter- 
mination of a valid block count is supplied, as a 
fixed value, to the comparator 82 through a line 
820. As the comparator 82, a general comparator 
such as an SN7485 available from Texas Instru- 
ments is used. If the valid block count supplied 
from the counter 81 is equal to or larger than the 
fourth threshold vaJue supplied through the line 
820. the comparator 82 outputs a signal designat- 
ing reset execution. If the valid block count is less 
than the fourth threshold value, the comparator 82 
outputs a signal designating a reset stop. The 
above-described logic is set in the comparator 82 
in advance. An output signal from the comparator 
82 is supplied, as an output from the valid block 
count determining section 10, to the valid block 
reset section 10 through a line 890. Fig. 22 shows 
an arrangement of the valid block reset section 1 1 . 
The valid block reset section 11 is constituted by a 
selector 91. The fourth valid block map is supplied 
from the isolated invalid block removing section 6 
to one input terminal of the selector 91 through the 
line 680. A value representing an invalid block is 
supplied to the other input terminal of the selector 
91 through a line 920. Subsequently, a signal in- 
dicating whether the valid block count is not less 
than the fourth threshold value is supplied, as a 
selection signal, from the valid block count deter- 
mining section 10 to the selector 91 through the 
line 890. If the selection signal supplied through the 
line 890 indicates that the valid block count is 
equal to or larger than the fourth threshold value, 
the selector 91 selects the signal representing the 
invalid block, which is supplied through the line 
920. If the selection signal supplied through the line 
890 indicates that the valid block count is less than 
the fourth threshold value, the selector 91 selects 
the fourth valid block map supplied through the line 
680. An output signal from the selector 91 is sup- 
plied, as the fifth valid block map, to the frame 
delay section 12 through a line 910. 

An apparatus based on the difference detection 
scheme according to the embodiment of Figs. 15 
to 19D will be described below with reference to 
Fig. 23. 

The valid block reset section 11 is omitted 
from the apparatus of this embodiment, and a valid 
block reset function is assigned to a weighting 
section 103. A valid block count determining sec- 
tion 10 counts valid blocks in the fourth valid block 
map supplied from an isolated invalid block remov- 
ing section 6, and performs threshold determination 
of the obtained valid block count by using the 
fourth threshold value, thus supplying to the weigh- 
ting section 103 a signal indicating whether the 
valid block count is the fourth threshold value or 
more. The weighting section 103 performs the sec- 



ond weighting operation of the fifth valid block map 
supplied from a frame delay section 12 to obtain 
the sixth valid block map. If the determination sig- 
nal supplied from the valid block count determining 

5 section 10 indicates that the valid block count is 
equal to or larger than the fourth threshold value, 
the weighting section 103 replaces all the valid 
blocks in the sixth valid block map with invalid 
blocks. In contrast to this, if the determination sig- 

ro nal indicates that the valid block count is less than 
the fourth threshold value, the weighting section 
103 outputs the sixth valid block map without any 
modification. The sixth valid block map output from 
the weighting section 103 is supplied to an adder 

T5 4. 

Furthermore, in the present invention, a speak- 
er region can be extracted by detecting the motion 
of the speaker between frames, and performing 
segmentation of a portion corresponding to the 

20 motion, instead of performing the above-described 
difference detection scheme. In this case, the mo- 
tion of a speaker between two consecutive frames 
must be detected. As a method of detecting such a 
motion between two consecutive frames, a method 

25 based on the principle of motion compensation is 
available. For example, a method described by 
Ninomiya, et al„ "Motion Compensation Inter-frame 
Coding Scheme", Shingakuron (B), J63 - B, 11, pp. 
1140 - 1147, November 1976 is known. In this 

30 method, each frame is divided into small blocks. 
The image portions of the previous frame are 
stored in units of blocks. Blocks, of the current 
frame, which have the highest correlation with the 
respective image portions of the previous frame are 

35 calculated, and the differences in position (motion 
vectors) between the corresponding blocks, and 
the differences in amplitude (motion-compensated 
predictive errors) between pixels spatially located 
at the same position between the corresponding 

40 blocks are transmitted. The motion-compensated 
vectors may be obtained by a method other than 
the method described above. 

A method of extracting a speaker region by 
using the above-mentioned motion-compensated 

45 vectors according to the present invention will be 
described in detail below with reference to the 
accompanying drawings. Assume that a speaker 
moves as indicated by the frames at time to, time 
t1 , and time t2 in Fig. 1 . When the motion between 

so the frames at time t1 and time t2 is compensated, 
motion vectors indicated by arrows in Fig. 24 are 
obtained. In this case, if the motion between the 
frames at time t2 and time t3 occurs mainly at a 
mouth portion, the motion vectors indicated by 

55 arrows in Fig. 25A are obtained. An isolated arrow 
portion in the background portion corresponds to a 
motion vector generated by noise in the back- 
ground. A block in which a motion vector is gen- 
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erated is defined as a valid block, and a block in 
which no motion vector is generated is defined as 
an invalid block. Figs. 3B and 25B respectively 
show a valid block map between time t1 and time 
t2 and a valid block map between time t2 and time 
t3, which are obtained by the above-described pro- 
cessing. Referring to Figs. 3B and 25B, shaded 
portions represent valid blocks. Assume that Fig. 
3A shows the fourth valid block map obtained 
between time to and time tl. The valid block map 
of the current frame, i.e., the first valid block map, 
is subjected to the first weighting operation, where- 
as the valid block map of the previous frame, i.e., 
the fourth valid block map, is subjected to the 
second weighting operation. The weighted valid 
block maps of the current and previous frames are 
added/synthesized to obtain the second valid block 
map. Fig. 4A shows the second valid block map at 
time t2, which is obtained by the above-descried 
weighting operations. Subsequently, segmentation 
of the second valid block map in Fig. 4A is per- 
formed. In the segmentation of the first coding 
method using motion vectors, valid and invalid 
blocks are determined by using the second thresh- 
old value, as described in the method based on the 
difference detection scheme. 

In the second coding method using motion 
vectors, different segmentation operations are per- 
formed depending on whether the ratio of a valid 
block count of the previous frame, i.e., a valid block 
count of the fourth valid block map, as a denomina- 
tor, to a valid block count of the valid block map of 
the current frame, i.e., a valid block count of the 
first valid block map, as a numerator, is not less 
than the fourth threshold value or less than it. If the 
valid block count is equal to or larger than the 
fourth threshold value, segmentation is performed 
by using the second threshold value in the same 
manner as in the first method. 

Fig. 4B shows the third valid block map ob- 
tained by the above-described segmentation in the 
first and second methods of coding a moving im- 
age signal according to the present invention. In 
the third valid block map, an isolated invalid block 
may be generated. In this case, blocks adjacent to 
the isolated invalid block are referred. If the values 
of the neighboring blocks are equal to or larger 
than a third predetermined threshold value, the 
value of the isolated invalid block as a target is 
replaced with a value representing a valid block. 
With the above-described processing, the region 
corresponding to the isolated invalid block in Fig. 
4B is removed to obtain the fourth valid block map. 
Fig. 6 shows the fourth valid block map from which 
the isolated invalid block is removed. 

A segmentation operation in the second coding 
method according to the present invention will be 
described below with reference to Figs. 6, 25, 26, 



and 27. Assume, in this case, that the ratio of a 
valid bock count of the previous frame to a valid 
block count of the current frame is less than the 
first threshold value, similar to the case at time t3. 
5 When the fourth valid block map obtained at time 
t2, shown in Fig. 6. and the first valid block map 
obtained at time t3, shown in Frg. 25 B, are respec- 
tively subjected to the first and second weighting 
operations and are synthesized with each other, the 
w second valid block map shown in Fig. 26A is ob- 
tained. When segmentation of the second valid 
block map is performed in accordance with the 
second threshold value, the third valid block map 
indicated by a shaded portion in Fig. 26B is ob- 
75 tained. Subsequently, an isolated invalid block re- 
moving operation of the third valid block map is 
performed in accordance with the fourth threshold 
value to obtain the fourth valid block map. In this 
case, since no isolated invalid block is present in 
20 the third valid block map obtained at time t3, the 
fourth valid block map is identical with the third 
valid block map. Subsequently, only a portion, of 
the moving image signal, which corresponds to the 
valid block region in the fourth valid block map is 
25 coded. However, as shown in Fig. 26B, the chest 
portion and upper right head portion of the speaker 
are omitted from the valid block region of the fourth 
valid block map. If, therefore, coding is performed 
in this state, non-coded regions may be formed at 
30 the chest portion and the head portion, and dis- 
continuous portions may be formed in the speaker 
region of the coded image. This may deteriorate 
the coded image. For this reason, if the ratio of a 
valid block count of the previous frame to a valid 
35 block count of the current frame is less than the 
first threshold value, and the valid block count of 
the current frame is small, as in the case at time t3, 
the threshold value in segmentation is switched to 
prevent omissions from the speaker region. When, 
40 for example, segmentation is executed with respect 
to the weighted second valid block map shown in 
Fig. 26A, if the block k has a value other than "0", 
the threshold value in segmentation is set to be 
small enough to make the segmentation target 
45 block k valid. With this operation, the third valid 
block map shown in Fig. 27 can be obtained, and 
omissions from the speaker region can be pre- 
vented. The threshold value in this segmentation is 
defined as the fifth threshold value. As described 
so above, if the ratio of a valid block count of the 
previous frame to a valid block count of the current 
frame is equal to or larger than the fourth threshold 
value, segmentation is performed by selecting the 
second threshold value. In contrast to this, if the 
55 ratio of a valid block count of the previous frame to 
a valid block count of the current frame is less than 
the fourth threshold value, and the valid block 
count of the current frame is much smaller than the 
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valid block count of the previous frame, segmenta- 
tion is performed by selecting the fifth threshold 
value. Subsequently, the moving image signal cor- 
responding to the valid block region shown in Fig. 
27, i.e., the speaker region, is coded by using 
either or both of inter-frame correlation and intra- 
frame correlation. 

A method of extracting a speaker region in the 
third coding method using motion vectors will be 
described in detail below with reference to the 
accompanying drawings. Assume that a speaker 
moves as indicated by the frames at time tO. time 
t1, and time t2 in Fig. 1. When motion vectors 
between the frames at time tO and time t1 are 
obtained, regions indicated by arrows in Fig. 28 is 
obtained. Assume, in this case, that the isolated 
arrow portion in the background portion is a motion 
vector generated by noise in the background, and 
that blocks in which motion vectors are generated 
are defined as valid blocks, and blocks in which no 
motion vectors are generated are defined as invalid 
blocks. Fig. 29B shows a valid block map at time 
n, which is obtained by the above-described pro- 
cessing. Referring to Fig. 29B, shaded portions are 
valid blocks. Assume that Fig. 29 A shows the sixth 
valid block map obtained between a frame at tO 
and a frame at time tO-1 immediately preceding the 
frame at time tO. The first weighting operation is 
performed with respect to the valid block map (Fig. 
29B) of the current frame, i.e., the first valid block 
map, whereas the second weighting operation is 
performed with respect to the valid block map (Fig. 
29A) of the previous frame, i.e., the sixth valid 
block map. The weighted valid block maps of the 
previous and current frames are added/synthesized 
to obtain the second valid block map. Fig. 29C 
shows the second valid block map. 
Subsequently, segmentation of the synthesized 
second valid block map shown in Fig. 29C is 
performed. 

Fig. 29D shows the third valid block map ob- 
tained by segmentation. An isolated invalid block 
may be formed at a moving portion in the third 
valid block for the following reason. When the first 
valid block map is obtained, no motion vector is 
detected in a block whose difference value relative 
to a corresponding block of the previous frame is 
slightly smaller than the threshold value for motion 
vector detection, i.e., a block in which only a slight 
luminance change occurs and a simple pattern is 
formed. Such a block becomes an invalid block. As 
a result, an isolated invalid bock is sometimes 
formed at a moving portion. Fig. 30 shows a case 
wherein an isolated invalid block x is formed. The 
isolated invalid block x shown in Fig. 30 is removed 
by an isolated invalid - block removing operation to 
obtain the fourth valid block map. Since no isolated 
invalid block is present in the third valid block map 



in Fig. 29 D, the fourth valid block map is identical 
with the third valid block map shown in Fig. 29D. 
Processing at time t2 will be described next. 
Fig. 31A shows the first valid block map 

5 formed by obtaining motion vectors between 
frames at time t1 and time t2. The first weighting 
operation of the first valid block map is performed. 
The second weighting operation of the fourth valid 
block at time t1 shown in Fig. 29D is performed. 

to When the first and fourth valid bock maps, which 
respectively have undergone the first and second 
weighting operations, are added/synthesized, the 
second valid block shown in Fig. 31 B is obtained. 
When the above-described segmentation of the 

75 second valid block map in Fig. 31 B is performed, 
the third valid block map shown in Fig. 31 C is 
obtained. Subsequently, an isolated invalid block 
removing operation of the third valid block map is 
performed. Since no isolated invalid block is 

20 present in the third valid block map in Fig. 31 C, the 
third valid block map is used as the fourth valid 
block. The shaded portion in the fourth valid block 
map is the speaker region obtained by segmenta- 
tion. Although the actual speaker region at time t2 

25 corresponds to substantially the left half of the 
frame, a large portion of the speaker region ob- 
tained by segmentation extends to the background 
portion in the right half of the frame. If, therefore, 
the fourth valid block map in Fig. 31 C is used 

30 without any modification, noise in the background 
may be undesirably coded. That is, when the mo- 
tion of a speaker is large as in the case of the 
frames at time t1 and time t2, and the number of 
valid blocks obtained by segmentation is large, the 

35 valid block region of the current frame extends into 
the speaker region of the previous frame due to the 
influences of the valid block map of the previous 
frame. If, therefore, the motion of the speaker be- 
tween frames is large, i.e., the number of valid 

40 blocks in the fourth valid block map is equal to or 
larger than a third predetermined threshold value, a 
reset operation is performed with respect to the 
fourth valid block map to replace all the valid 
blocks in the fourth valid block map with invalid 

45 blocks to obtain the fifth valid block map. The fifth 
valid block map is delayed by a one-frame time to 
become the sixth valid block map. The six valid 
block map is subjected to segmentation at the next 
timing. Assume that Fig. 29A shows the fourth valid 

so block map of the previous frame, and that Fig. 29B 
shows the valid block map of the current frame, 
i.e., the first valid block map. If the number of valid 
blocks in the fourth valid block map obtained at 
time t1 is equal to or larger than the fourth thresh- 

55 old value, all the valid blocks in the fourth valid 
block map are replaced with invalid blocks to ob- 
tain the fifth valid block map. Therefore, ail the 
valid blocks in the sixth valid block map at time t2, 
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which is obtained by delaying the fifth valid block 
may by a one-frame time, are replaced with invalid 
blocks. If the resulting first valid block map ob- 
tained at time t2 is the valid block map shown in 
Fig. 31 A, the weighted second valid block map is 
the valid block map shown in Fig. 31 D. If seg- 
mentation of this second valid block map is per- 
formed, the third valid block map shown in Fig. 
31 A can be obtained. Since no isolated invalid 
block is included in this third valid block map. the 
third valid block map becomes the fourth valid 
block map without any modification, thus removing 
the background portion. 

Fig. 32 shows the first coding apparatus using 
motion vectors according to the embodiment of 
Figs. 24 to 31 D. The apparatus shown in Fig. 32 is 
identical with the apparatus in Fig. 7 except that a 
motion vector detecting section 101 is arranged in 
place of the validity determining section 1. The 
same reference numerals in Fig. 32 denote the 
same parts as in Fig. 7, and a description thereof 
will be omitted. An input moving image signal is 
supplied to the motion vector detecting section 101 
and a delay section 9 through a line 10. The 
motion vector detecting section 101 stores the 
moving image signal of the previous frame, and 
divides the new moving image signal input through 
the line 10 into blocks, each consisting of a plural- 
ity of pixels, i.e., n horizontal pixels x n vertical 
pixels. In addition, the motion vector detecting sec- 
tion 101 calculates blocks having the highest cor- 
relation with the respective image portions of the 
previous frame stored in units of blocks, and ob- 
tains motion vectors representing the differences 
between the corresponding blocks. In this case, 
blocks in which motion vectors are generated are 
defined as valid blocks, whereas blocks in which no 
motion vectors are generated are defined as invalid 
blocks, thereby obtaining the first valid block map. 
The first valid block map obtained by the motion 
vector detecting section 101 is output to a weigh- 
ting section 2. 

Fig. 33 shows the second coding apparatus 
using motion vectors according to this embodiment 
of the present invention. Similar to the apparatus 
shown in Fig. 32, the apparatus shown in Fig. 33 is 
obtained by replacing the validity determining sec- 
tion 1 of the apparatus shown in Fig. 11 with a 
motion vector detecting section 101. In this appara- 
tus, operations subsequent to vector detection are 
performed in the same manner as in the apparatus 
shown in Fig. 1 1 . 

Fig. 34 shows the third coding apparatus using 
motion vectors according to this embodiment of the 
present invention. The apparatus shown in Fig. 34 
is identical with the apparatus shown in Fig. 30 
except that a motion vector detecting section 101 
is arranged in place of the validity determining 



section 1. 

Fig. 35 shows a modification of the apparatus 
using a motion vector, in which the arrangement of 
a coding unit 107 is simplified by adding a delay 

5 section 13. The delay section 13 performs delay 
compensation with respect to motion vectors sup- 
plied from a motion vector detecting section 101 to 
establish synchronization with the fourth valid block 
map. The time-compensated motion vectors output 

io from the delay section 13 are supplied to the 
coding unit 107 through a line 970. As shown in 
Fig. 36, the motion vector detecting section 71 in 
Fig. 14 is omitted from the coding unit 107. In the 
coding unit 107, the motion vectors supplied from 

rs the delay section 13 through the line 970 are 
supplied a frame memory 77 and a variable-length 
coding unit 78. 

Fig. 37 shows a modification of the apparatus 
using the motion vectors, in which a delay section 

20 13 is added to the apparatus shown in Fig. 34, and 
a coding unit 1 07 is used. 

Fig. 38 shows a modification of the apparatus 
shown in Fig. 33. 

Fig. 40 shows a modification of the apparatus 

25 in Fig. 23, in which the validity determining section 
1 in Fig. 23 is replaced with a motion vector 
detecting section 101, and a delay section 13 and 
a coding section 107 are used. 

Fig. 39 shows an arrangement of the ratio 

30 determining section 8 shown in Figs. 11, 33, and 
38. The ratio determining section 8 is constituted 
by counters 81 and 82, and a ROM 83. The coun- 
ter 81 of the ratio determining section 8 counts 
valid blocks in the first valid block map supplied 

35 from the motion vector detecting section 1 through 
a line 120. The counter 82 of the ratio determining 
section 8 counts valid blocks in the fourth valid 
block map supplied from the isolated invalid block 
removing section 6 through a line 680. Valid block 

40 count data output from the counters 81 and 82 are 
supplied to the ROM 83. The ROM 83 determines 
whether the ratio of the valid block count of the 
fourth valid block map to the valid block count of 
the first valid block count, which are obtained by 

45 the counters 81 and 82, is not less than a first 
predetermined threshold value. In the ROM 83, a 
value representing that the ratio of the valid block 
count of the fourth valid block map to the valid 
block count of the first valid block map is equal to 

so or larger than the third threshold value is written 
beforehand in an address area in which the ratio is 
equal to or larger than the first threshold value, and 
a value representing that the ratio is less than the 
third threshold value is written beforehand in an 

55 address area in which the ratio is less than the first 
threshold value. A signal representing whether the 
ratio is not less than the first threshold value, which 
is output from the ROM 83, is supplied, as an 
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output from the ratio determining section 8, to the 
segmentation section 5 through the line 85. 

As has been described in detail above, in the 
method and apparatus for coding a moving image 
signal according to the present invention, since 
only a speaker region obtained by segmentation is 
coded, unnecessary information generated by 
noise in a background portion can be omitted, and 
the coding efficiency can be improved. 

Claims 

1. A method of coding a moving image signal, 
characterized by comprising the steps of: 

detecting a change between a current 
frame and a previous frame in units of blocks 
obtained by dividing a frame constituted by an 
input moving image signal into a plurality of 
pixels; 

forming a first valid block map in units of 
frames by determining valid and invalid blocks 
on the basis of the detected changes; 

performing a first weighting operation of 
the formed first valid block map; 

performing a second weighting operation 
of a fourth valid block map of the previous 
frame; 

obtaining a weighted second valid block 
map by adding/synthesizing the first and fourth 
valid block maps respectively subjected to the 
first and second weighting operations; 

obtaining a third valid block map by per- 
forming predetermined segmentation of the 
second valid block map; 

obtaining a fourth valid block map by de- 
termining validity of an isolated invalid block in 
the third valid block map by referring to neigh- 
boring blocks; and 

coding a valid block region of the fourth 
valid block map. 

2. A method according to claim 1, further com- 
prising the step of delaying the moving image 
signal by an amount corresponding to a time 
interval between the instant at which the mov- 
ing image signal is input and the instant at 
which the fourth valid block map is formed, 
and wherein the step of coding is the step of 
coding the fourth valid block map of the de- 
layed moving image signal. 

3. A method according to claim 1 or 2, wherein a 
difference between the current and previous 
frames is detected as a detected change 
therebetween, and the first valid block map is 
formed such that if the detected difference is 
not less than a predetermined first threshold 
value, a corresponding block is defined as a 



valid block, and if the detected difference is 
less than the first threshold value, a corre- 
sponding block is defined as an invalid block. 

5 4. A method according to claim 1 , 2, or - 3, 
wherein a motion between the current and 
previous frames is detected as the detected 
change therebetween, and the first valid block 
map is formed such that a block in which a 

w motion is detected is defined as a valid block, 

and a block in which no motion is detected is 
defined as an invalid block. 

5. A method according to any one of claims 1 to 
75 4. wherein a motion vector representing the 

direction and magnitude of a motion between 
the current and previous frames is detected as 
the detected change therebetween in units of 
blocks, and the first valid block map is formed 
20 such that a block in which a motion vector is 

detected is defined as a valid block, and a 
block in which no motion vector is detected is 
defined as an invalid block. 

25 6. A method according to any one of claims 1 to 

5, wherein the segmentation is performed such 
that if values of a target block and neighboring 
blocks are not less than a second predeter- 
mined threshold value, the target block is de- 

30 fined as a valid block, and rf the values are 

less than the second threshold value, the tar- 
get block is defined as an invalid block, there- 
by obtaining the third valid block map. 

35 7. A method according to any one of claims 1 to 

6, wherein if values of blocks adjacent to the 
isolated invalid block in the third valid block 
map are not less than a third predetermined 
threshold value, the isolated invalid block is 

40 replaced with a valid block, and if the values 

are less than the third threshold value, the 
isolated invalid block is kept used as an invalid 
block. 

45 a A method according to any one of claims 1 to 

7, wherein the fourth valid block map is coded 
by using at least one of inter-frame correlation 
and intra-frame correlation. 

so 9. A method according to any one of claims 1 to 

8, wherein the segmentation is performed to 
obtain the third valid block map on the basis of 
a ratio of a valid block count of the fourth valid 
block map to a valid block count of the first 

55 valid block map. 

10. A method according to claim 9. wherein when 
the ratio between the valid block counts is not 
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less than a fourth predetermined threshold val- 
ue, if values of a target block and neighboring 
blocks are not less than a second predeter- 
mined threshold value, the target block is de- 
fined as a valid block, and if the ratio is less 
than the second threshold value, the target 
block is defined as an invalid block, thereby 
obtaining the third valid block map from the 
second valid block map. 

11. A method according to claim 9, wherein when 
the ratio between the valid block counts is not 
less than a fourth predetermined threshold val- 
ue, if values of a target block and neighboring 
blocks are not less than a fifth predetermined 
threshold value, the target block is defined as 
a valid block, and if the ratio is less than the 
fifth threshold value, the target block is defined 
as an invalid block, thereby obtaining the third 
valid block map from the second valid block 
map. 

12. A method according to any one of claims 1 to 
11, wherein a fifth valid block map is obtained 
from the fourth valid block map on the basis of 
a valid block count of the fourth valid block 
map. 

13. A method according to claim 12, wherein if the 
valid block count of the fourth valid block map 
is not less than a sixth predetermined thresh- 
old value, all valid blocks in the fourth valid 
block map are replaced with invalid blocks, 
thereby obtaining a fifth valid block map, and if 
the valid block count is less than the sixth 
threshold value, the fourth valid block map is 
used as a fifth valid block map without any 
modification. 

14. A method according to claim 12 or 13, wherein 
the fifth valid block map is delayed by a one- 
frame time to obtain a sixth valid block map, 
and the second weighting operation of the 
sixth valid block map in place of the fourth 
valid block map is performed. 

15. An apparatus for coding a moving image sig- 
nal, characterized by comprising: 

forming means (1. 101) for detecting a 
change between a current frame and a pre- 
vious frame in units of blocks obtained by 
dividing a frame constituted by an input mov- 
ing image signal into a plurality of pixels, and 
forming a first valid block map in units of 
frames by determining valid and invalid blocks 
on the basis of the detected changes; 

first weighting means (2) for performing a 
first weighting operation of the formed first 



valid block map; 

second weighting means (3) for performing 
a second weighting operation of a fourth valid 
block map of the previous frame; 
5 adding means (4) for obtaining a weighted 

second valid block map by adding/synthesizing 
outputs from said first and second weighting 
means; 

segmentation means (5) for obtaining a 
w third valid block map by performing predeter- 

mined segmentation of the second valid block 
map; 

isolated invalid block removing means (6) 
for obtaining a fourth valid block map by deter- 
75 mining validity of an isolated invalid block out- 

put from said segmentation means by referring 
to neighboring blocks; and 

coding means (7) for coding a valid block 
region output from said isolated invalid block 
20 removing means. 

16. An apparatus according to claim 15, further 
comprising input signal delay means (9) for 
delaying the input moving image signal to es- 
25 tablish synchronization with the fourth valid 

block map, and wherein said coding means 
codes a valid block of an output from said 
input signal delay means. 

30 17. An apparatus according to claim 15 or 16, 
wherein said forming means is validity deter- 
mining means (1) for detecting a difference 
between the current and previous frames, and 
forming the first valid block map such that if 

35 the detected difference is not less than a first 

predetermined threshold value, a correspond- 
ing block is defined as a valid block, and if the 
detected difference is less than the first thresh- 
old value, a corresponding block is defined as 

40 an invalid block. 

18. An apparatus according to claim 15, 16, or 17, 
wherein said forming means detects a motion 
between the current and previous frames, and 

45 forming the first valid block map such that a 

block in which a motion is detected is defined 
as a valid block, and a block in which no 
motion is detected is defined as an invalid 
block. 

50 

19. An apparatus according to any one of claims 
15 to 18, wherein said forming means is mo- 
tion vector detecting means (101) for detecting 
a motion vector representing the direction and 

55 magnitude of a motion between frames in units 

of blocks, and forming the first valid block map 
such that a block in which a motion vector is 
detected is defined as a valid block, and a 
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block in which no motion vector is detected is 
defined as an invalid block. 

20. An apparatus according to any one of claims 
15 to 19. wherein said segmentation means 
obtains the third valid block map such that if 
values of a target block and neighboring blocks 
are not less than a second predetermined 
threshold value, the target block is defined as 
a valid block, and if the values are less than 
the second threshold value, the target block is 
defined as an invalid block. 

21. An apparatus according to any one of claims 
15 to 20, wherein said isolated invalid block 
removing means replaces the isolated invalid 
block with a valid block when values of neigh- 
boring blocks are not less than a third pre- 
determined threshold value, and keeps the iso- 
lated invalid block to be an invalid block when 
the values are less than the third threshold 
value. 

22. An apparatus according to any one of claims 
15 to 21, wherein said coding means perform- 
ing a coding operation by using at least one of 
inter-frame correlation and intra-frame correla- 
tion. 

23. An apparatus according to any one of claims 
15 to 22, further comprising ratio determining 
means (8) for determining whether a ratio of a 
valid block count of the fourth valid block map 
to a valid block count of the first valid block 
map is not less than a fourth predetermined 
threshold value, and wherein said segmenta- 
tion means obtains a third valid block map in 
accordance with an output from said ratio de- 
termining means. 

24. An apparatus according to claim 23, wherein 
when the ratio between the valid block counts 
is not less than the fourth threshold value, said 
segmentation means obtains a third valid block 
map from the second valid block map such 
that if values of a target block and neighboring 
blocks are not less than a second predeter- 
mined threshold value, the target block is de- 
fined as a valid block, and if the values are 
less than the second threshold value, the tar- 
get block is defined as an invalid block. 

25. An apparatus according to claim 23, wherein 
when the ratio between the valid block counts 
is not less than the fourth threshold value, said 
segmentation means obtains a third valid block 
map from the second valid block map such 
that if values of a target block and neighboring 



blocks are not less than a fifth predetermined 
threshold value, the target block is defined as 
a valid block, and if the values are less than 
the fifth threshold value, the target block is 
5 defined as an invalid block. 

26. An apparatus according to any one of claims 
15 to 25, further comprising valid block count 
determining means (10) for determining wheth- 

w er a valid block count of the fourth valid block 

map is not less than a sixth predetermined 
threshold value, and valid block reset means 
(11) for obtaining a fifth valid block map from 
the fourth valid block map on the basis of an 

75 output from said valid block count means. 

27. An apparatus according to claim 26, wherein 
said valid block reset means replaces all valid 
blocks in the fourth valid block map with in- 

20 valid blocks when the valid block count of the 

fourth valid block map is not less than the sixth 
threshold value to obtain a fifth valid block 
map, and sets the fourth valid block map as a 
fifth valid block map without any modification 

25 when the valid block count is less than the 

sixth threshold value. 

2& An apparatus according to claim 26 or 27, 
further comprising frame delay means (12) for 

30 obtaining a sixth valid block map by delaying 

the fifth valid block map by a one-frame time, 
and wherein said second weighting means per- 
forms the second weighting operation of the 
sixth valid block map in place of the second 

35 valid block map. 

29. An apparatus according to any one of claims 
15 to 25, further comprising valid block count 
determining means (10) for determining wheth- 

40 er a valid block count of the fourth valid block 

map is not less than a sixth predetermined 
threshold value, and frame delay means (12) 
for obtaining a fifth valid block map by delay- 
ing the fourth valid block map by a one-frame 

45 time, and wherein said second weighting 

means is means (103) for supplying a sixth 
valid block map obtained by performing a re- 
set operation of the fifth valid block map on the 
basis of an output from said valid block count 

so determining means after the fifth valid block 

map from said frame delay means is weighted. 

30. An apparatus according to any one of claims 
19 to 29, further comprising motion vector de- 

55 lay means (13) for delaying a motion vector 

from said motion vector detecting means to 
establish synchronization with the fourth valid 
block map. 
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